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Item analysis has essential roles in the learning assessment. The item analysis 
program is designed to measure student achievement and instructional 
effectiveness. This study was aimed to develop item-analysis program and verify 
its feasibility. This study uses a Research and Development (R & D) model. The 
procedure includes designing and developing a product, validating, and testing the 
product. The data were collected through documentations, questionnaires, and 
interviews. This study successfully developed item analysis program, namely 
AnBuso. It is developed based on classical test theory (CTT). It was practical and 
applicable for Indonesian teachers to analyse test items. 

Key Words: feasibility, item-analysis program, AnBuso, development, practical item, 
teachers 

INTRODUCTION 

Learning processes determine the quality of education. This implies that teachers have a 
very central role in improving the quality of education. They have crucial roles not only 
in designing and implementing but also in assessing the learning processes and their 
outcomes. Improving the quality of learning and assessment system is one of the efforts 
to improve the quality of education. 

Learning and assessments can be said to be ‘two sides of the same coin’ (Guliker, 
Bastianes, & Kirschner, 2004). All forms of assessment are essential components of 
teaching (Russell & Airasian, 2012) and they have strong influences on the quality of 
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learning (Raymond, Homer, Smith, & Gray, 2012). Assessments involve not only 
finding out what students have learned but also improving the processes of learning and 
teaching (Bers, 2008). This implies that assessments have essential roles in improving 
not only the learning processes but also the learning outcomes. Good assessments will 
encourage teachers to determine appropriate instructional strategies and motivate 
students to study harder. 

An assessment is a process of gathering evidence of student learning to inform 
instructional decisions (Stiggins & Chappuis, 2012). It means that teachers have to 
gather accurate evidence of the achievements of their students as the quality and impacts 
of their instructional decisions depend on it. The development of assessment instruments 
should therefore measure all aspects of learning objectives. Good students should attain 
better scores than the poor ones. 

The implementation of the new curriculum, known as the 2013 curriculum, in the 
secondary education levels in Indonesia is facing many problems (Republika, 2014), for 
example the difficulty of changing teachers’ mind-set (Metronews, 2014; Tempo, 2013). 
A study reported that 87 percent of Indonesian teachers had difficulties in understanding 
the implementation of authentic assessments (Susilowati, 2013) as the assessment 
system includes too many aspects that must be measured (Tempo, 2014). It means that 
their ability to perform good assessments has to be improved. 

An assessment is a systematic process of providing credible evidence of resources, 
implementation, and learning outcomes with the purpose of improving and developing 
effective instruction, programs, and services in higher education (Banta & Palomba, 
2014). Thus, it is a process of gathering information about the students and in turn the 
information is employed to make instructional decisions and improve learning 
processes. 

An assessment should be able to assess students' progress. Such information can be used 
as a basis for identifying the students’ status in the group and determining the next steps 
of learning. Therefore, the assessment has to be able to assess their competence 
accurately. Assessments help teachers clarify the goals and the purposes of their 
teaching, create learning experiences in real-life contexts, and provide many different 
ways for students to demonstrate their abilities and skills (Darling-Hammond, 2014). 
Properly used assessment procedures can contribute directly to the improvement of 
learning, such as (1) clarifying the nature of the intended learning outcomes, (2) 
providing short-term goals to work toward, (3) providing feedback concerning learning 
progress, (4) providing information for overcoming learning difficulties and for 
selecting future learning experiences, and (5) identifying the next instructional goal 
(Miller, Linn, & Gronlund, 2009). 

The purposes of assessments include establishing a classroom that supports learning, 
planning and conducting instruction, placing students, providing feedback, diagnosing 
student problems and disabilities, and summarizing and grading academic learning and 
progress (Russell & Airasian, 2012). Thus, assessments are used to monitor the learning 
progress, provide information as a basis for giving feedback, and improve learning. 
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To provide accurate information, assessments have to be based on several criteria, such 
as valid, reliable, competence focused, comprehensive, objective, and educative 
(Puskur, 2008). The three concepts—validity, reliability, and objectivity—have to do 
with the quality of the information obtained from tests or other assessment instruments 
or methods (Anderson, 2003; Kubiszyn & Borich, 2013). Thus, the validity and 
reliability are important parts of assessments to obtain appropriate and accurate 
information. 

Item analyses play a somewhat more important role in the construct and predictive 
validation (Nunnally & Bernstein, 1994). Item analysis is the process of examining the 
students’ responses to each test item, to judge the quality of the item (Mehrens & 
Lehmann, 1991). There are two approaches to analyze item for the test development, the 
Classical Test Theory (CTT) and Item Response Theory (IRT) (Crocker & Algina, 
2008). They use difficulty level and discriminant index to determine the item quality. It 
means that the item analysis is an important process, which determines the validity of 
items and therefore it has an important role in identifying good items. The assessment 
results also provide accurate information about students' abilities therefore they can help 
teachers to identify which materials are difficult. The item analysis also provides 
information about the materials that have not been mastered by the students so that 
teachers can plan remedial activities. Based on the reasons it is necessary to develop 
practical program that can help teachers analyse items practically. 

There are many software that develop to analyze item, based on either CTT or IRT, such 
as ITEMAN (Berk & Griesemer, 1976) based on CTT, and BILOG-MG, MULTILOG, 
PARSCALE and TESTFACT based on IRT (Croudace, Ploubidis, & Abbot, 2005). 
However, they have not been used optimally yet by Indonesian teachers because most 
software uses foreign languages which are difficult for them to understand. Some 
software is also less practical and applicable because it produces output with different 
formats. Therefore, it is necessary to develop more practical and applicable software 
that can cater for their needs. This present study aims to develop item analysis software 
or program that could be used by Indonesian teachers to prepare learning administration 
reports. It is named AnBuso. It is a program developed in Microsoft Excel. This 
program uses features, functions and formulas available on Microsoft Excel, so it only 
works with Microsoft Excel. 

AnBuso is developed based on CTT. CTT has dominated the area of standardized 
testing and an advantage with CTT is relatively easy to interpret (Wiberg, 2004). CTT 
assumed that the observed score is seen as an estimate of the true scores plus/minus 
some unobservable measurement error (Allen & Yen, 1979; Crocker & Algina, 2008; 
Hambleton & Swaminathan, 1985). 

CTT uses the difficulty level and discriminant index to determine the quality of the item. 
Item difficulty is the percentage of students who answer an item correctly. Generally, 
item difficulties of about .3 to .7 maximize the information the test provides about 
differences among examinees (Allen & Yen, 1979: 121; Kaplan & Saccuzzo, 2005: 
170). Item discrimination refers to the ability of an item to differentiate among students. 
The item discrimination index provided by AnBuso is a Pearson Product Moment 
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correlation between item score and total scores. This index is the equivalent of a point- 
biserial coefficient in this application. Arbitrary interpretations have been made on 
ranges of values of the point-biserial correlation. They range from Very Good items 
(>0.40), Good items (<0.39, >0.30), Fair/marginal items (<0.29, >0.20), Non¬ 
discriminating/poor items (<0.19, >0.00), Need attention (<0.00) (Alagumalai & Curtis, 
2005: 8; Ebel & Frisbie, 1991: 232; Crocker & Algina, 2008: 315). It means that the 
item with correlation coefficient less than 0.2 has to be rejected. Based on these criteria, 
an item is considered good if it has difficulty index between 0.3 to 0.7 and item 
discriminant coefficient more than 0.2. AnBuso uses these criteria to judge a good or 
poor item. 

METHOD 

This study uses a model of Research and Development (R & D). The procedure includes 
designing and developing a product, validating, and trying the product. A preliminary 
study was carried out to determine the ability and willingness of teachers to use item 
analysis and identify their real needs. The next step was designing a model of 
application program and its guidebook, which were validated by experts and revising 
them in order to obtain validated products. 

The product validation was conducted using the Delphi method. This validation 
processes also involved some experts in education, measurements and computer 
programming. The testing of the trial products were conducted twice: limited and 
extended testing. The test was to obtain comprehensive information and input from 
users. The results show that all the instruments, the program and its guidebook are 
feasible to use although there are some suggestions for revision. The results of this test 
were also used to improve the quality of the products so that they are ready to be 
disseminated widely. 

The study involved 65 respondents comprising teachers and school supervisors from 
five districts in Yogyakarta (DIY). Purposive sampling was carried out to select the 
respondents with computer literacy, especially the ability to operate Microsoft Excel 
program. 

Observation, documentation, questionnaires, and interviews were employed to elicit the 
research data. Observation was conducted to obtain data or information about the 
teachers’ ability to use the application program that has been developed. The data of 
instruments used to measure students' abilities, such as formative and summative tests 
were obtained by documentation techniques. The elicited documents took the forms of 
final examination tests, daily tests, remedial programs, and the results of the previous 
item analysis used by the teachers. 
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Table 1 


Feasibility Categories 


No 

Score 

Category 

1 

More than M + 1,8 SD 

Very feasible 

2 

M + 0,6 SD toM + 1,8 SD 

Feasible 

3 

M - 0,6 SD to M + 0,6 SD 

Fair 

4 

M - 1,8 SD to M-0,6 SD 

Less feasible 

5 

Less than M - 1,8 SD 

Not feasible 


The questionnaire elicited feedback or suggestion from teachers, supervisors, education 
department officers, and education experts. The questionnaire was also developed to 
measure the feasibility of the program concerning its visual display, contents, 
usefulness, practicality, and convenience. Interviews were conducted to teachers, 
supervisors, education department officers, and experts to obtain information regarding 
the strength and drawbacks of the AnBuso program. 

The data were analysesed descriptively and supported by tables and graphs. Five scales 
of categories were employed to measure the feasibility of the program (see Table 1). 

FINDINGS 

The study involved 65 participants comprising teachers, supervisors and education 
actors. Most of them (72%) were teachers of high schools (SMA, SMK, and MA). 
Many of them (68%) were teachers of public schools and 57% were government 
employees. Respondents came from various fields of study, including Economics, 
Accounting, Mathematics, Social Studies, English, Indonesian, Information Technology, 
Chemistry, Physics, Biology, Geography, Arabic, and so on. 

The willingness of the teachers to analyze test items was low (57%). Only 11% of 
them regularly conducted test item analysis, 20% of them often did it and 12% of them 
never did it. It would be important to assess why the teachers had a low willingness to 
analyze test items. 

In general, the teachers would perform item analysis if the supervisors ordered them to 
do so. It means that their awareness to perform item analysis was very low. This is 
because they were unable to use the available program. Of course this contributed to the 
poor quality of test items they have developed. 

This study successfully developed AnBuso program and its manual book. AnBuso was 
developed on Microsoft Excel program. It has been improved based on the feedback 
from the respondents of the present research. The AnBuso was accompanied with a 
manual book in Indonesian language that will help teachers take advantage of the 
program. It provides step-by-step instructions which are very practical for the teachers 
to operate the program. The book contains an introduction, contents, input and data 
report. 

The program has undertaken some important modifications in order to be compatible 
with the assesments system of the 2013 curriculum. Some revisions were conducted to 
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improve its visual display, InputOl spreadsheet, Participant Reports spreadsheet, 
Participant Remedial spreadsheet, and formula changes. The modifications on the 
visual displays had to be done because it had too many color variations, which have 
made the application less attractive. The adjustments were done not only on the input 
spreadsheet, but also on the report spreadsheet (see Figure 1). They were made based on 
the respondents’ feedback. 


AnBuso 5.0 

©2011-2014 by Ali Muhson 

IDENTITAS 


|(llll.L,U Jip-,la.LLL,Ll.LLIl -tof 

u nurnghupu, •*** d ~ ““-' ^ 

Data Umum 

Kolom Penglslan 

VALIDASI 

ISahnnPenddikan 

SMA Neqeti 80 Ynnvakarta 


Mata PeUiaran 

Ekoeom 

OK 


X11PS 

OK 

Nama les 

Sumatif 

OK_ 

SK/KD 



Nama Gunj 

Ali Muhson 

OK 

WP 

196811121999031003 

OK 

Semester 

Gasal 

OK 

lahim Pelaiaran 

2012/2013 

OK 

Tamjgal Tes 

17 Agustus 7012 

OK 

Tanaoal Openksa 

7 Seotember 2012 

OK 


Mustafa 

OK 

MP Kepaa Sekoah 

1972031111984031003 

^ -1 


Previous visual display 

Figure 1 


AnBuso 5.1 For Teacher 

©2011-2014 by Ali Muhson 


s NIP Keoala Sekolah 


1972031111984031003 


i Skala Penilaian (4.10 atau 100) 


Input 01 Input 02 l 


New visual display 


The spreadsheet of InputOl before and after modifications 


AnBuso 5.1 For Teacher 

© 2011-2014 by Ali Muhson 



(Hanya diperkenankan mengisi data atau menghapus tetapi tidak boleh memindah isi data atau 
menggunakan fasilitas Cut Paste) 


Keterangan 

Kolom Pengisian 

VALIDASI 

Tpmnat 1 apnran 

Yogyakarta 

OK 

Tanggal Laporan 

20 Agustus 2014 

OK 

Skala Penilaian (4,10 atau 10^1^ 

4 

OK 

"RTTai KKM 

2.66 

OK 


Jumlah dan Bobot Soal 

Jumlah soal pilihan ganda (Max 50) 

20 

OK 

Jumlah soal essay (Max 10) 

5 

OK 

Bobot soal pilihan ganda 

60% 

OK 

Bobot soal essay 

40% 

OK 


Data Soal Pilihan Ganda 


11 imldh Altornatif Icmiahan /May M M I 

InputOl Input 02 Lap Peserta Lap Butir Pola Jawaban Butir ||| Lap Essay Materi Remidial Pese 


Figure 2 
Grade revision 

AnBuso has been adjusted to assessment system of the 2013 curriculum of the 
secondary education levels in Indonesia so that it accommodates Indonesian teachers’ 
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needs. Some amendments were also made on the assessment scales. In the previous 
model, the grading scales were 1-10 and 1-100. To cater for the assessments system of 
the 2013 curriculum, the new model provides a 1-4 grade scale (see Figure 2). 

Basically the purpose of the item analysis is to determine the quality of the test items 
and obtain information about the assessment results so that teachers can make a follow¬ 
up plan such as remedial and enrichment teaching. The program also provides 
information about the students who are included in the remedial and enrichment group. 

Figure 3 shows the general overview of the AnBuso system. Basically the program is 
divided into three components: input, process and output. Data input consists of Input 
01 which takes general data, and Input 02 which contains the identity of students and 
their answers. The data entered into the Input 01 spreadsheet include the School Name, 
Subject, Class/Program, Name of test, SK/KD, Name of the Teacher, Teacher’s NIP, 
Semester, Academic Year, Test Date, Inspected Date, name of the school Principal, 
Principal’s NIP, Reporting Place, Reporting Date, Grades (10 or 100), the score of 
Minimum competence criteria (KKM), Objective Test Data, and Essay Test Data. The 
data entered into the Input 02 spreadsheet include Students Name, Gender (M = Male, 
F = Female), Multiple Choice Answer Sheet, and Essay Score. Whenever the data are 
entered in the input 01 and input 02 spreadsheets, the program will automatically 
generate a complete report consisting of: 



Figure 3 

Flowchart Diagram Revision 
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1. Student Report Sheet which displays list of test scores. The list contains the name 
of student and gender, number of items answered correctly and incorrectly, 
objective test score, essay test score, final score, and description (passed and 
failed), The other information in this sheet is the total number of the students, the 
number (percentage) of students who passed the tests, the minimum scores, 
maximum scores, average scores, and the standard deviation. 

2. Item Report Sheet which shows the result of Multiple Choice Analysis. It contains 
Discriminant Index, levels of difficulty, Malfunction Distractors, and Conclusion of 
item in multiple choice test. 

3. Response Pattern Report Sheet which contains Distribution of the answers of 
Multiple Choice tests. 

4. Essay Report Sheet that contains Discriminant Index, levels of difficulty, and 
Conclusion of item in essay test. 

5. Remedial Material Reports Sheet which contains materials for Individual and 
Classical Remedial tests. 

6. Remedial Group Reports Sheet which contains the grouping of participants who 
take remedial tests. 

7. Graphics Sheet which contains charts of minimum competence criteria (KKM) 
score distributions, and the proportions of the learning mastery. 



Figure 4 

The results of the feasibility test 


The results of the feasibility test showed that AnBuso program, developed in this study, 
was considered very feasible by the respondents. Of the total respondents, 51% said 
that it is feasible, 46% stated it is very feasible and the rest stated it is quite feasible. 
This shows that the program has a high usefulness in helping teachers perform item 
analysis. 
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The aspect that gets the highest score is its practicality or convenience, and usefulness. 
The lowest rated aspect is the visual graphic display (see Figure 4). As the respondents 
reported, the AnBuso is very practical to assist them analysing test items and it really 
provides them with complete information. The program is able to analyze not only 
multiple choice test but also essay tests. The results of the test-item analysis are 
presented in the forms of report, therefore it is very easy for teachers to interpret them. 

CONCLUSION 

This study successfully developed AnBuso based on CTT, which can be used by 
Indonesian teachers to analyse items practically. It is developed on Microsoft Excel 
program. It contains data input spreadsheet, data processing spreadsheet, and analysis 
report sheet in tables and chart. AnBuso is considered as very feasible program 
regarding its practicality and convenience, usefulness, and contents, while its screen 
display is considered only feasible. As AnBuso is very feasible and very useful, it 
should be disseminated more widely to Indonesian teachers. It needs to be upgraded 
periodically in order to fulfil teachers’ future need. 
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Turkish Abstract 

Endonezyali Ogretmenler I^in Uygulamali Madde Analizi 

Madde analizi ogrenmenin degerlendirilmesinde onemli bir role sahiptir. Madde analiz programi 
egitimin etkililigini ve ogrenci ba§arismi olgmek igin uygulamr. Bu gali§ma bir madde analiz 
programi geli§tirerek programm uygulanabilirligini ogmeyi amaglami§tir. £ah§mada, Ara§tirma 
ve Geli§tirme (R&D) modeli kullamlmi§tir. l§lem basamaklan, uriin dizaym ve geli§tirilmesi, 
uriinun onaylanmasi ve test edilmesi a§amalarmdan olu§maktadir. Veriler belgeleme, anketler ve 
g6ra§meler yoluyla toplanmi§tir. Bu gah§mada, ba§arili bir §ekilde AnBuso adi verilen klasik test 
teorisine (CTT) dayali bir madde analiz programi geli§tirilmi§tir. Program Endonezyali 
ogretmenlerin madde analizi yapmalan igin uygun ve uygulanabilir bulunmu§tur. 

Anahtar Kelimeler: uygunluk, madde analiz programi, AnBuso, ogretmenler, ogrenmenin 
degerlendirilmesi, ogrenci ba§ansi, egitimin etkililigi 


French Abstract 

Le Developpement de Programme d’Analyse d’Article Pratique pour Professeurs 
indonesiens 

L'analyse d'article a des roles essentiels dans 1'evaluation apprenant. Le programme d'analyse 
d'article est congu pour mesurer l'accomplissement d'etudiant et l'efficacite d'instruction. Cette 
etude a ete visee a se developper l'analyse d'article programme et verifie sa faisabilite. Cette etude 
utilise une Recherche et developpement (la R et D) le modele. La procedure inclut la conception 
et le developpement d'un produit, une validation et un test du produit. Les donnees ont ete 
rassemblees par des documentations, des questionnaires et des entretiens. Cette etude a avec 
succes developpe le programme d'analyse d'article, a savoir AnBuso. II est developpe base sur la 
theorie classique de test (CTT). C'etait pratique et applicable pour des professeurs indonesiens 
pour analyser des articles de test items. 

Mots Cles: faisabilite, programme d'analyse d'article, AnBuso, professeurs, apprenant evaluation, 
accomplissement d'etudiant, efficacite d'instruction 


Arabic Abstract 

. ~(._Jail Aidl ^ A ^^9 LluoLuoI 


(R & diaall ajlA ^,1-s'ntn .aljJLa. (j-o (JjjLaolIj jjj A'ill Ahll jJllI ajlA c. Vlg'ij 

(j-a djULidl £-a^« (j-a ^cILall ^ l 

(JjjLuoI OjJjlaJ ^aJJ AnBUSO. (s&J JjA'i £LaU^)J jJjlaJ ^ i" ~1 )jlA j t C—lUUfLai VI j 

. jLiLi.V! (Jalaai <Llij 4_iLac. (CTT). 

4j,ajU"lll Aailxiilj 4i_L/Uall jAnBuSO Vill didl £t-aU^)J dlLaKlI 
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German Abstract 

Die Entwicklung des Praktischen Itemanalyseprogramms fur Indonesische Lehrer 

Itemanalyse hat wesentliche Rollen in der Lernbewertung. Das Itemanalyseprogramm ist darauf 
ausgelegt, die Schulerleistung und die Lehrwirkung zu messen. Diese Studie zielte darauf ab, 
Item-Analyse-Programm zu entwickeln und seine Machbarkeit zu uberpriifen. Diese Studie nutzt 
ein Forschungs- und Entwicklungsmodell (R & D). Das Verfahren beinhaltet das Entwerfen und 
Entwickeln eines Produkts, die Validierung und das Testen des Produkts. Die Daten wurden 
durch Dokumentationen, Fragebogen und Interviews gesammelt. Diese Studie erfolgreich 
entwickelt Item Analyse-Programm, namlich AnBuso. Es basiert auf der klassischen Testtheorie 
(CTT). Es war praktisch und anwendbar far indonesische Lehrer, Testartikel zu analysieren. 

Schliisselworter: machbarkeit, item-analyse-programm, anbuso, lehrer, lernbewertung, 
schulerleistung, lehrwirkung 


Malaysian Abstract 

Pembinaan Program Analisis Item Praktikal untuk Guru Indonesia 

Analisis item mempunyai peranan penting dalam penilaian pembelajaran. Program analisis item 
bertujuan untuk mengukur pencapaian pelajar dan keberkesanan pengajaran. Kajian ini bertujuan 
untuk membangunkan program item-analisis dan mengesahkan kemungkinan itu. Kajian ini 
menggunakan model penyelidikan dan pembangunan (R & D). Prosedur ini termasuk mereka 
bentuk dan membangunkan produk, mengesahkan, dan menguji produk. Data yang telah 
dikumpulkan melalui dokumen, soal selidik, dan temu bual. Kajian ini berjaya membangunkan 
program analisis item, iaitu AnBuso. Ia dibangunkan berdasarkan teori ujian klasik (CTT). Ia 
adalah praktikal dan digunakan untuk guru Indonesia untuk menganalisis item ujian. 

Kata Kunci: kemungkinan, program item-analisis, AnBuso, guru-guru, penilaian pembelajaran, 
pencapaian pelajar, keberkesanan pengajaran 


Russian Abstract 

Pa3pa6oTKa IIpaKTHHecKoro AHajuoa fljm HHAOHe3HHCKHx yuHTejieu 

AHajiH3 AHajiH3 nrpaeT Ba^Hyio pom> b oijeHice odyHemra. AHamrs npe/jMeTOB npe/jHa3HaneH 
/yifl BMHBJiemiH ycneBaeMocTH ynamnxcfl h 3(j)(j)eKTHBH0CTH odynemoL 3to HCCJie/joBamie 6bijio 
HanpaBjieHO Ha pa3pa6oTKy aHajiH3a npe/jMeTOB h npOBepKy ee oeymecTBHMOCTH. B 3tom 
uccjie^OBaHHH Hcnom>3yeTc>i Moment Hccne/jOBaHHH h pa3pa6oTOK (R & D). npoije/jypa 
BKjHonaeT b ceda npoeKTHpOBaHne h pa3pa6oTKy npo^yicra, npOBepKy h TecTHpOBamie 
npo^yKTa. ^aHHbie 6bijih codpaHbi c noMomtio ^OKyMeHTai^HH, aHKeT(BonpocHHKOB) h 
HHTepBbio. Oho ycnemHO pa3pa6oTajio aHajiH3 nporpaMMy, a hmchho AnBuso. IIporpaMMa 
pa3pa6oTaHa Ha ocHOBe KjiaccHnecKOH TeopHH tcctob (CTT). 3to CTano npaKTHHHbiM h 
npHMeHHMbiM yuHTen^MH HH£OHe3HH ^jui aHajiH3a nyHKT tcctob. 

KmoneBbie CnoBa: bbhiojihhmoctb, aHajiH3 nporpaMM, AnBuso, yuHTejui, oijemca odyneHua, 
yeneBaeMOCTB ynamHxcji, s^eKTHBHOCTb odyneHmi 
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